Repository navigation
feat(billing): settle at the tier the request's input length selects - #417
Merged
Merged
Conversation
Models can declare optional `[[models.tiers]]`: price steps chosen by the input token count of a single request — cache-miss plus cache-hit prompt tokens, not the user's cumulative usage and not the output. Step 1 landed the field and its validation (496a723) but left settlement untouched, so every request was billed at the first step. This step carries the steps into the database (`models.tiers_json`, schema v17) and settles at the selected one; a model without steps, or a request below the first `from_tokens`, keeps billing at the model's own single price — byte for byte as before. `from_tokens` is an inclusive lower bound (`[from_tokens, next)`). Vendors that publish an open interval are expressed by adding 1 in the data: MiniMax's "input >512k" → 512001, xAI's and Google's "prompt exceeds 200k tokens" → 200001. Tier data is filled only where the vendor publishes a number, verified against the pricing pages: minimax-m3 (>512k), gemini-3.1-pro, grok-4.6 and grok-4.7 (>200k). OpenAI splits short/long context without publishing the boundary, Anthropic's pricing page has no long-context step, and qwen3.7-flash publishes input/output per step but no cache-hit price, so those rows stay single-priced and are noted as pending in the config. The admin model editor clears `tiers_json` on write: it is a config-owned field, and a hand-edited base price would otherwise leave stale steps charging long prompts, so manual edits fall back to the single price (the next start re-seeds it from config). Tested: cargo test 579/0, fmt/clippy/non-test build clean. Boundary legs at from_tokens−1 / from_tokens / from_tokens+1, a settle-level leg that reads the recorded points back across the 200k boundary, and a leg fixing the selection to cache-miss + cache-hit. rant 2026-10-16T14:08:14.003832+08:00 (tiered price and cache-write price)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Models can now declare optional
[[models.tiers]], and settlement bills a request at the price step its input length selects.Step 1 (
496a723) landed the field and its validation but left settlement untouched, so every request was billed at the first step. This step:models.tiers_json(schema v16 → v17), written bydb::seed_modelsfrom config and read back bydao::get_model_pricein the same query it already ran;billing::select_tier, on this request's input tokens (cache-miss + cache-hit prompt tokens), falling back to the model's own single price when there are no steps or the input is below the firstfrom_tokens;peak_*is a model-level trio and applies on top of the selected step.from_tokensis an inclusive lower bound ([from_tokens, next)). Vendors that publish an open interval are expressed by adding 1 in the data: MiniMax's "input >512k" →512001, xAI's and Google's "prompt exceeds 200k tokens" →200001.Tier data is filled only where the vendor publishes a number (checked against the pricing pages on 2026-10-16/17):
minimax-m3gemini-3.1-progrok-4.6,grok-4.7Rows deliberately left single-priced: OpenAI splits short/long context but does not publish the boundary token count; Anthropic's pricing page has no long-context step;
qwen3.7-flashpublishes input/output per step but no cache-hit price, so writing a step would drop its cache-hit price to 0 (under-charging). Each is noted as pending, with the reason, in the config header.The admin model editor now clears
tiers_jsonon write: it is a config-owned field, and a hand-edited base price would otherwise leave stale steps charging long prompts. Manual edits fall back to the single price, and the next start re-seeds the row from config.Cache-write price (
cache_write_input_per_m) stays declaration-only in this step, and the config comment now says so: no upstream response path yields a cache-write token count today, so wiring it into billing needs an extra usage parse.Related Issue
References rant
2026-10-16T14:08:14.003832+08:00(tiered price and cache-write price), step 2 — settlement. Step 1 (496a723) landed the expression and validation.Changes
config/config.example.toml) and the database roster (docs/architecture.md)Tests
cargo testall green — 579 passed / 0 failed (8 new)cargo fmt --checkpassescargo clippy --all-targets -- -D warningsand a non-testcargo buildwith-D warningsare cleanNew legs:
select_tierboundaries atfrom_tokens−1/from_tokens/from_tokens+1(and past the last step); a settle-level test that drives the real path (seed_models→dao→select_tier→ ledger) and reads the recorded points back across the 200k boundary; a leg fixing the selection basis to cache-miss + cache-hit; andtiers_jsonround-trip plus malformed-JSON fallback legs. Both boundary legs were shown to bite by mutating the implementation (dropping the cache-hit term; making the lower bound exclusive) and watching them go red.Checklist
feat/)